IceCube -- Neutrinos in Deep Ice The Top 3 Solutions from the Public Kaggle Competition

During the public Kaggle competition "IceCube -- Neutrinos in Deep Ice", thousands of reconstruction algorithms were created and submitted, aiming to estimate the direction of neutrino events recorded by the IceCube detector. Here we describe in detail the three ultimate best, award-winning solutions. The data handling, architecture, and training process of each of these machine learning models is laid out, followed up by an in-depth comparison of the performance on the kaggle datatset. We show that on cascade events in IceCube above 10 TeV, the best kaggle solution is able to achieve an angular resolution of better than 5 degrees, and for tracks correspondingly better than 0.5 degrees. These performance measures compare favourably to the current state-of-the-art in the field

The Sound Demixing Challenge 2023 \unicode{x2013} Cinematic Demixing Track

This paper summarizes the cinematic demixing (CDX) track of the Sound Demixing Challenge 2023 (SDX'23). We provide a comprehensive summary of the challenge setup, detailing the structure of the competition and the datasets used. Especially, we detail CDXDB23, a new hidden dataset constructed from real movies that was used to rank the submissions. The paper also offers insights into the most successful approaches employed by participants. Compared to the cocktail-fork baseline, the best-performing system trained exclusively on the simulated Divide and Remaster (DnR) dataset achieved an improvement of 1.8dB in SDR whereas the top performing system on the open leaderboard, where any data could be used for training, saw a significant improvement of 5.7dB.Comment: under revie

The Sound Demixing Challenge 2023 \unicode{x2013} Music Demixing Track

This paper summarizes the music demixing (MDX) track of the Sound Demixing Challenge (SDX'23). We provide a summary of the challenge setup and introduce the task of robust music source separation (MSS), i.e., training MSS models in the presence of errors in the training data. We propose a formalization of the errors that can occur in the design of a training dataset for MSS systems and introduce two new datasets that simulate such errors: SDXDB23_LabelNoise and SDXDB23_Bleeding1. We describe the methods that achieved the highest scores in the competition. Moreover, we present a direct comparison with the previous edition of the challenge (the Music Demixing Challenge 2021): the best performing system under the standard MSS formulation achieved an improvement of over 1.6dB in signal-to-distortion ratio over the winner of the previous competition, when evaluated on MDXDB21. Besides relying on the signal-to-distortion ratio as objective metric, we also performed a listening test with renowned producers/musicians to study the perceptual quality of the systems and report here the results. Finally, we provide our insights into the organization of the competition and our prospects for future editions.Comment: under revie

The MABe22 Benchmarks for Representation Learning of Multi-Agent Behavior

Real-world behavior is often shaped by complex interactions between multiple agents. To scalably study multi-agent behavior, advances in unsupervised and self-supervised learning have enabled many different behavioral representations to be learned from trajectory data. However, such representation learning approaches are generally evaluated on specific datasets and tasks, and it is difficult to compare methods quantitatively to measure progress on representations for behavior analysis. We aim to address this by introducing a large-scale, multi-agent trajectory dataset from real-world behavioral neuroscience experiments that covers a range of behavior analysis tasks. Our dataset consists of common model organisms (mice and flies) in a variety of settings (different strains, lengths of interaction, optogenetic and thermogenetic stimulation), with a subset consisting of expert-annotated behavior labels. Improvements on our dataset corresponds to behavioral representations that work across multiple organisms and is able to capture differences for common behavior analysis tasks. Sample Python notebooks and evaluator to use our dataset is available at: https://www.aicrowd.com/challenges/multi-agent-behavior-challenge-202